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Abstract. A floorplan is a tiling of a rectangle by rectangles. There are natural ways to order the elements — 
rectangles and segments — of a floorplan. Ackerman, Barequet and Pinter studied a pair of orders induced 
by neighborhood relations between rectangles, and obtained a natural bijection between these pairs and 
(2-41-3, 3-14-2)-avoiding permutations, also known as (reduced) Baxter permutations. 

In the present paper, we first perform a similar study for a pair of orders induced by neighborhood relations 
between segments of a floorplan. We obtain a natural bijection between these pairs and another family of 
permutations, namely (2-14-3, 3-41-2)-avoiding permutations. 

Then, we investigate relations between the two kinds of pairs of orders — and, correspondingly, between 
(2-41-3,3-14-2)- and (2-14-3, 3-41-2)-avoiding permutations. In particular, we prove that the superposition 
of both permutations gives a complete Baxter permutation (originally called w-admissible by Baxter and 
Joichi in the sixties). In other words, (2-14-3, 3-41-2)-avoiding permutations are the hidden part of complete 
Baxter permutations. We enumerate these permutations. To our knowledge, the characterization of these 
permutations in terms of forbidden patterns and their enumeration are both new results. 

Finally, we also study the special case of the so-called guillotine floorplans. 



1. Introduction 

A floorplar^ is a partition of a rectangle into interior-disjoint rectangles such that no point belongs to 
the boundary of four rectangles (Fig. [T]). We call segment of the floorplan any straight line, not included in 
the boundary of the partitioned rectangle, that is the union of some rectangle sides, and is maximal for this 
property For example, each of the floorplans of Fig. [1] has four horizontal and four vertical segments. Since 
four rectangles of a floorplan never meet, the segments do not cross, and a meeting of segments has one of 
the following forms: n, 1, H, T (but not + ). 
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Figure 1. Two R-equivalcnt floorplans. 



An easy induction shows that the number of segments in a floorplan is smaller than the number of rectangles 
by 1. Throughout the paper, for a given floorplan P, the number of segments in P is denoted by n; accordingly, 
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n + 1 is the number of rectangles in P. We say that P has size n+1. For instance, the floorplans in Fig. [TJ 
have size 9. 

Many papers have appeared about floorplans, not only in combinatorics but also in the computational 
geometry literature [131 1251 [28] . The interest in floorplans is motivated, in particular, by the fact that their 
generation is a critical stage in integrated circuit layout [23] [24] [3TJ [37] [38] , in architectural designs [8] [16] [T9] 

CHUGS!, etc. 

The present paper is combinatorial in nature, and describes the relationship between a pair of natural 
orders defined on the segments of a floorplan and certain pattern avoiding permutations. It parallels a 
previous paper of Ackerman, Barcquet and Pinter in which a similar study was carried out for a pair of 
orders defined on rectangles of a floorplan, in connection with the so-called Baxter permutations [TJ. These 
permutations were introduced in the sixties, and were originally called reduced Baxter permutations, as 
opposed to complete Baxter permutations [6] [7] [IT]. A complete Baxter permutation 7r has an odd size, say 
2n+l, and is completely determined by its values at odd points, 7r(l), 7r(3), . . . , 7r(2n + l). After normalization, 
these values give the associated reduced Baxter permutation 7r (the subscript 'o' stands for odd; we denote 
by 7r e the permutation obtained by normalizing the list 7r(2),7r(4), . . . ,7r(2n)). Our paper provides the even 
part of the theory initiated in [TJ: we prove that the permutation associated with the segments and the 
permutation associated with the rectangles are respectively the even and odd parts n e and tt of the same 
complete Baxter permutation n. We also characterize the even parts of Baxter permutations in terms of 
forbidden patterns and enumerate them. 

This satisfactory final picture is, however, not our original motivation for studying segments of floorplans. 
Instead, our starting point was the observation that many questions on floorplans deal with segments rather 
than rectangles. An interesting example is the rectangulation conjecture [2] Conj. 7.1]. It asserts that for any 
floorplan P of size n+1, and any set II of n points of the rectangle such that no two of them lie on the same 
horizontal or vertical line, there exists a floorplan P', R-equivalen10 to P, such that each point of II lies on a 
segment of P' (see Fig. [2] for an illustration). Such an embedding of points in segments of a floorplan is called 
a rectangulation. The rectangulation conjecture was suggested in connection with the minimization of the 
total length of segments in certain families of rectangulations [25]. Hence, the initial motivation of this paper 
was to provide us with a better combinatorial insight into problems related to floorplans and, in particular, 
to rectangulations. 



A floorplan P 



A point set II 



II embedded in P' 



Figure 2. A rectangulation: embedding a point set IT in a floorplan P. 



In order to present our results in greater detail, we first need to describe the related results obtained for 
rectangles in [JJ. In that paper, the authors study a representation of two order relations between rectangles 
in floorplans by means of permutations. These order relations arc induced by neighborhood relations, which 
arc defined as follows. A rectangle A is a left-neighbor of B (equivalcntly, B is a right-neighbor of A) if there 
is a vertical segment in the floorplan that contains the right side of A and the left side of B. (Note that the 
right side of A and the left side of B may be disjoint.) Now, the relation "A is to the left of B" (equivalcntly, 



This equivalence relation is defined further down in the introduction. 
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"B is to the right of A"), denoted by A «- B, is denned as the transitive closure of the relation 11 A is a 
left- neighbor of BP Finally, the relation «- is the reflexive closure of *-. The terms A is a below-neighbor 
of B (equivalently, B is an above-neighbor of A) and A is below B (equivalently, B is above A) arc defined 
similarly, as well as the notation A i B for " A is below i?,"0 and A { B for " A = B or A { B." It is easy to see 
that the relations «- and | are partial orders. In both floorplans of Fig. [1] we have, among other relations, 
Ail (because A is a below- neighbor of C, and C is a below-neighbor of I), A <- G, and B <- F. 

The following results are proved in [1]. Let P be a floorplan of size n + 1. Two distinct rectangles A and 
£> of P are in exactly one of the relations A *- B, B *- A, A J, B, or £? J, A. It follows that the relations ,/ 
and \ between rectangles of P defined by 

A / B if A = B , or A is to the left of B, or A is below B, 
A \ B if A = B , or A is to the left of B, or A is above B, 
are linear orders (the signs / and \ arc intended to resemble the inequality sign ^). Each of these orders 
can be used to label the rectangles of P by 1,2, ...,n + 1. In the / order, the rectangle in the lower left 
corner is labeled 1, and the rectangle in the upper right corner n+1. In the \ order, the rectangle in the 
upper left corner is labeled 1, and the rectangle in the lower right corner n + 1. Let R(P) be the sequence 
a\ ai ... a n +i, where, for all 1 < i < n+ 1, Oj is the label in the ^ order of the rectangle which is labeled i in 
the \ order. It is clear that R(P) is a permutation of [n + 1] := {1,2, . . . , n+ 1}; we call it the R- permutation 
of P. Loosely speaking, R(P) is obtained by labeling the rectangles according to the / order, and then 
reading these labels while passing the rectangles according to the \ order. Fig. [3] shows the construction of 
the R-permutation of a floorplan P; the right part of the figure is the graph of p = R(P), that is, the point 
set {(i,p(i)) ■■ l<i<n + l}. 
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Figure 3. Constructing the R-pcrmutation of a floorplan P. 

Two floorplans P\ and Pi of size n + 1 are said to be R-equivalenf^ if there exists a labeling of the 
rectangles of Pi by A\, A2, . . . , A n+ \ and a labeling of the rectangles of P2 by B\, B2, . ■ . , B n+ i such that for 
all k,m e [n + 1], the rectangles Ak and A m exhibit the same neighborhood relation as and B m . The 
two floorplans in Fig. Q] are R-cquivalent: this follows from the labeling presented in this figure (in fact, 
A,B,C, ... I is the / -order). It is easy to prove that two floorplans are R-equivalent if and only if they have 
the same R-permutation. 

The main results of Q] state that for any floorplan P, its R-permutation is a (2-41-3, 3- 14-2) -avoiding per- 
mutatiorQ, originally called (reduced) Baxter permutation; moreover, R is a bijection between R-equivalence 
classes of floorplans and (2-41-3, 3-14-2)-avoiding permutations. Through this correspondence, the size of a 

B 

"^Hence, A J. B should be understood as I ; and similarly for ^. 

A 

%n QQ, two R-equivalent floorplans are actually treated as two representations of the same floorplan. 
^This notation is explained in Section 13.31 
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floorplan becomes the size of the permutation, and the order relations between rectangles in P can be easily 
read from R(P). 

We can now describe our results in greater detail. 

In the first part of the paper, we develop for segments a theory that parallels the theory developed for 
rectangles in [I]. We define two order relations between segments (Section [2]), which leads to the notion of S- 
equivalent floorplanfl Then we use these orders to construct a permutation S(P) called the S '-permutation of 
P. In Section[3]we prove that S-permutations coincide with (2-14-3, 3-41-2)-avoiding permutations, and that 
S, regarded as a function from S-equivalence classes to (2-14-3, 3-41-2)-avoiding permutations, is a bijection. 
The description of S and S^ 1 are fairly simple (both can be constructed in linear time), but as often, the 
proof remains technical, despite our efforts to write it carefully. 

In the second part of the paper, we super-impose our theory with the analogous theory developed for 
rectangles in [I] . In Section 2] we show that R-equivalence of floorplans implies their S-equivalence (this 
means that the R-equivalcnce refines the S-equivalence), and explain how S(P) can be constructed directly 
from R(P). This construction shows that S(P) and R(P), combined together, form the so-called complete 
Baxter permutation associated with i?(P), as defined in the seminal papers on Baxter permutations [6|l7llllj. 
We also describe in terms of R when two floorplans give rise to the same S-pcrmutation. This is another 
difficult proof, but we need this description to express the number of (2-14-3, 3-41-2)-avoiding permutations 
in terms of the number of Baxter permutations (Section [5]) . 

To finish, in Section [6] we characterize and enumerate S-permutations corresponding to the so-called guil- 
lotine floorplans; a similar study was carried out in [T] for R-permutations. We end in Section [7] with a few 
remarks. 

2. Orders between segments of a floorplan 

In this section we define neighborhood relations between segments of a floorplan, use them to define 
two partial orders (denoted «- and {) and two linear orders (denoted / and \), and prove several facts 
about these orders. Most of them are analogous to facts about the orders on rectangles mentioned in the 
introduction, and proved in [1]. 

Definition 2.1. Let P be a floorplan. Let I and J be two segments in P. We say that I is a left-neighbor 
of J (cquivalently, J is a right-neighbor of /) if one of the following holds: 

• I and J are vertical, and there is exactly one rectangle A in P such that the left side of A is contained 
in / and the right side of A is contained in J; 

• / is vertical, J is horizontal, and the left endpoint of J lies in /; or 

• / is horizontal, J is vertical, and the right endpoint of I lies in J. 

The terms "J is a below-neighbor of J" (equivalently, "J is an above-neighbor of I") are defined similarly. 

Typical examples are shown in Fig. [4] 





/ / 








/ 






J I 




J I 




J I 


J I 


J 









Figure 4. The segment / is a left- neighbor of the segment J. 



'In the notion of R-equivalence and S-equivalcncc, R stands for rectangles and S for segments. 
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Note that a horizontal segment I has at most one left-neighbor and at most one right-neighbor (no such 
neighbor(s) when the corresponding endpoint(s) of / lie on the boundary), which are both vertical segments. 
In contrast, a vertical segment may have several left- and right-neighbors, which may be horizontal or vertical. 
This is illustrated in Fig. [5] 





















./ 







Figure 5. The right-neighbors of a vertical segment I (thick segments). Note that the 
vertical segment J is not a right-neighbor of /. 



Definition 2.2. The relation "I is to the left of J" (equivalently, J is to the right of I), denoted by I <- J, 
is the transitive closure of the relation "I is a left-neighbor of J." The relation «- is the reflexive closure of 
*-. The relations 1 1 J ("7 is below J") and I \ J (for "I = J or I J, J") are defined similarly. 

Observation 2.3. The relations «- and { are partial orders. 

Proof. We prove the claim for the relation «-. Rcflcxivity and transitivity follow from the definition. For 
antisymmetry, note that I *- J and J *- I cannot hold simultaneously because if I *- J, then any interior 
point of / has a smaller abscissa than any interior point of J. □ 

The following observation may help to understand the «- order. If I and J are vertical segments and right- 
left neighbors, let us create a horizontal edge, called traversing edge, in the rectangle A that lies between them. 
Fig. [6] shows a chain of neighbors in the «- order, starting from a vertical segment J, and the corresponding 
traversing edges (dashed lines). 



/ 



Figure 6. A chain in the «- order (thick segments), and the corresponding traversing edges 
(dashed lines). 

Observation 2.4. Assume I «- J. Then any point of J lies weakly to the right of any point of I (that is, its 
abscissa is at least as large). 

Let x (respectively, y) be a point of minimal (respectively, maximal) abscissa on I (respectively, J). Then 
there exists a polygonal line from x to y formed of vertical and horizontal sections, such that 
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- each vertical section is part of a vertical segment of the floorplan P, 

- each horizontal section is an (entire) horizontal segment of P, or a traversing edge of P, visited from 
left to right, 

- if I ( respectively, J ) is horizontal, it is entirely included in the polygonal line . 

It suffices to prove these properties when J is a right-neighbor of /, and they are obvious in this case (see 
Fig. [J). 

Lemma 2.5. In the «- order, J covers I if and only if J is a right-neighbor of I. A similar statement holds 
for the | order. 

Proof. Since «- is constructed as the transitive closure of the left-right neighborhood relation, every covering 
relation is a neighborhood relation. 

Conversely, let us prove that any neighborhood relation is a covering relation. Equivalently, this means that 
the right-neighbors of any segment I form an antichain. If / is horizontal, it has at most one right-neighbor, 
and there is nothing to prove. Assume / is vertical (as in Fig. [5j) , and that two of its neighbors, J\ and J 2 , 
satisfy 3\ *- J 2 . By the first part of Observation 12. 4[ J 2 cannot be horizontal (its leftmost point would then 
lie on /, leaving no place for J\). Hence J 2 is vertical. The possible configurations of I and J2 are depicted 
in the first four cases of Fig. 2] Let x (resp. y) be a point of / (rcsp. J2). By Observation 12.41 there exists 
a polygonal line from x to y that visits a point of J\ . This rules out the third and fourth cases of Fig. [4] 
(the line would be reduced to a single traversing edge). By symmetry we can assume that / and J2 are as 
in the first case of Fig. 0J Then the polygonal line, which is not a single traversing edge, has to leave / at a 
point that lies lower than the lowest point of J2 , and to reach J 2 at a point that lies higher than the highest 
point of J: this means that it crosses two horizontal segments, which is impossible given the description of 
this line. □ 

Lemma 2.6. Let I and J be two different segments in a floorplan P. Then exactly one of the relations: 
I *- J , J *- I, 1 1 J, or J { I , holds. 

Proof. Assume without loss of generality that / is a horizontal segment. Construct the NE-sequence K\, K 2 , ■ ■ ■ 
of / as follows (see Fig. [7] for an illustration): K\ is the right- neighbor of /, K 2 the above- neighbor of K\, K3 
the right-neighbor of K 2 , and so on, until the boundary is reached. Construct similarly the SE-, NW-, and 
SW-sequences of /. These sequences partition the rectangle into four regions (or fewer, if some endpoints of 
I lie on the boundary) ; each segment of P (except I and those belonging to either of the sequences) lies in 
exactly one of them. Also, if J is in the interior of a region, then its neighbors are either in the same region, 
or in one of the sequences that bound the region. 

It is not hard to see that the vertical segments of the NE-sequence are to the right of /, while horizontal 
segments are above /. A horizontal segment K 2 i cannot be to the left of /, since it ends to the right of /. Let 
us prove that K 2 i cannot be the right of / cither. Assume this is the case, and consider the polygonal line 
going from the leftmost point of / to the rightmost point of K 2 i, as described in Observation 12.41 The last 
section of this line is K 2 i. Hence the line has points in the interior of the region comprised between the NW- 
and NE-sequences. But since the line always goes to the right, and follows entirely every horizontal segment 
it visits, it can never enter the interior of this region. Thus K 2 i cannot be to the right of /. Thus, its only 
relation to / is I J, K 2 i . Similar arguments apply for vertical segments of the NE-sequence, and for the other 
three sequences. 

Consider now a segment J that lies, for instance, in the North region (that is the region bounded by the 
NE-sequence, the NW-sequence, and the boundary; the case of other regions is similar). Then / is below J: 
if we consider the below-neighbors of J, then their below-neighbors, and so on, then we necessarily reach one 
of the horizontal segments of the NW- or NE-sequence, which, as we have seen, are above I (we cannot reach 
a vertical segment of the sequences without reaching a horizontal segment first). 

Hence, we have that / J. J; it remains to prove that the other three relations are impossible. 
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First, J J, I is impossible since the relation J, is antisymmetric. To prove that J cannot be to the right of 
/, we argue as we did for K^c the polygonal line from I to J starting from the leftmost point of I cannot 
enter the North region. Symmetrically, J cannot be to the left of /. This completes the proof. □ 



J K 4 

1 * 3 

K 2 . 



Figure 7. Four regions determining the relationship between / and other segments of the floorplan. 

Definition 2.7. The relations / and \ between segments of a floorplan P arc defined by: 
I / J if I = J , or I is to the left of J, or I is below J, 
I \ J if I = J , or I is to the left of J , or I is above J. 

We also write I / J when I ^ J and I + J; and I \ J when I \ J and I + J. 

Proposition 2.8. The relations / and \ are linear orders. 

Proof. We prove the claim for the relation / . Reflcxivity follows from the definition. Antisymmetry follows 
from the fact that •«- and \ are order relations, and from Lemma 12.61 

For transitivity, assume that I / J and J / K . If I <- J and J <- K (respectively, 1 1 J and J | K), then 
we have I K (respectively, / | K) by the transitivity of <- (respectively, |). Assume now that I ■*- J and 
J [ K (the case 1 1 J and J <- K is proven similarly). By Lemma [2T6l / = K is impossible, and we have either 
I «- K, K <- J, I \ K, or K [ I. However, the combination of K ■*- I and I «- J yields A' <- J, contradicting 
the assumption that J { K (by Lemma l2.6[) . Similarly, combining K 1 1 with J J, K yields J { I, contradicting 
the assumption that I <- J. Therefore, we have either / <- K or / I K; in particular, / / K . 

Linearity follows from Lemma 12.61 □ 

Observation 2.9. The orders «- and | can be recovered from / and \ . Indeed, I •«- J if and only if I / J 
and I \ J; moreover, I { J if and only if I / J and J \ I. 

Throughout the paper, the ith segment in the \ order (1 < i < n) will be denoted by Ii. Sec Fig. 1111 for 
examples. 

We now explain how to determine ij+i among the neighbors of Ii. By Lcmma l2.51 Zj+i is either a right- or 
below-neighbor of h . There are several cases depending on the existence of these neighbors and the relations 
between them. For a horizontal segment /, we denote by R(/) the right- neighbor of / (when it exists). By 
Lemma 12. 51 the below-neighbors of / form an antichain of the \ order. Since \ is a linear order, they are 
totally ordered for the «- order. By the first part of Observation ^. 4[ the leftmost is also the smallest, denoted 
LB (J). Thus LB(I) is either LVB(I) (the leftmost vertical below-neighbor of I) or LHB(J) (the leftmost 
horizontal below-neighbor of /). Similarly, for a vertical segment /, we denote by B(/) the below-neighbor of 
/; by UR(7) the highest right-neighbor of /, and by UHR(I) (respectively, UVR(/)) the highest horizontal 



The letter U stands for up. 
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(respectively, vertical) right-neighbor of I. Fig. [8] illustrates the following observation (it is assumed that all 
candidates for ij+i are depicted. The dashed lines belong to the boundary). 

Observation 2.10. Let li be a segment in a floorplan P of size n + 1. If li is horizontal, then 7j + i is either 
R(/i) or LB (h). More precisely, 

(1) If none ofK(Ii) and LB(ii) exists, then li is the last segment in the \ order (that is, i = n). 

(2) If exactly one ofK(Ii) and LB(7i) exists, then F+i is this segment. 

(3) //LVB(ii) exists, then 7j + i =LB(/i). This segment is LHB(ii) if it exists, and otherwise LVB(/i). 

(4) //LVB(ii) does not exist but LHB(/;) andK(Ii) exist, then 

• If the join o/LHB(/i) and R(/i) is of type -\, then Jj+i = LHB(/, ). 

• If the join o/LHB(/j) and R(/i) is of type 1, then I i+ i = R(ii). 

If li is vertical, then Ii + \ is either B(/i) ; UHR(/, ), or UVR(ii) (the details are similar to those in the 
case of a horizontal segment). 
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Figure 8. The segment Jj+i follows li in the \ order (Top: li is horizontal. Bottom: li is vertical). 

One can in fact construct simultaneously, and in a single pass, the labeling of rectangles and segments. 

Proposition 2.11. Let P be a floorplan of size n + l, and let Ak denote the rectangle labeled k in the \ order. 
For 1 < k < n, the following property, illustrated in Fig. holds: 

• If the segments forming the SE-corner of Ak have a 1 join, let Jk be the segment containing the right 
side of Ak- Then Ak+i is the highest rectangle whose left side is contained in Jk- 

• If the segments forming the SE-corner of Ak have a H join, let Jk be the segment containing the lower 
side of Ak- Then Ak+i is the leftmost rectangle whose upper side is contained in Jk- 

In both cases, Jk is the fcth segment in the \ order of segments, denoted so far by Ik- 

Proof. By definition of the \ order, Ak+i is either a right-neighbor or a below-neighbor of Ak- If there is a 1 
join in the SE-corner of Ak, then all the right- neighbors of Ak are above all its below- neighbors. Therefore, 
Ak+i is the topmost among them. If there is a H join in the SE-corner of Ak, then all the below-neighbors of 
Ak are to the left of all its right-neighbors. Therefore, Ak+i is the leftmost among them. 

To prove the second statement, we observe it directly for k = 1, and proceed by induction. One has to 
examine several cases, depending on whether the segments in the SE-corners of Ak and of Ak+i have 1 or 
n joins. In all cases, Jk+\ is found to be the immediate successor of Jk in the \ order, as described in 
Observation 12 . 101 See Fig. [TO] for several typical situations. □ 

The group of symmetries of the square acts on floorplans (when fioorplans arc drawn in a square). It is thus 
worth examining how the orders are transformed when applying such symmetries. As this symmetry group 
is generated by two generators, for instance the reflections in the first diagonal and across a horizontal line, 
it suffices to study these two transformations. The following proposition easily follows from the description 
of the neighborhood relations of Fig. [4] 
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Figure 9. The rectangle A k+ i follows A k in the \ order. 



4+i 


4 


A k 


A k + i 




A k+2 







4 


A k 


A k +1 


A k+2 




4+i 











4+i 


A k+2 




4 


A k 


A , , 

k+l 



FIGURE 10. Successors of segments and rectangles for the \ orders. 

Proposition 2.12. Let P be a (square) floorplan, and let P' be obtained by reflecting P in the first diagonal. 
If I is a segment of P, let I' denote the corresponding segment of P' . Then 

1 1 j o r 
i / j ^ r / j' } 
i \ j ^ j r \ p. 

If instead P' is obtained by reflecting P in a horizontal line, 

I «- J ^ I 1 ^ J\ 

I{J *> J'\I\ 
I j/ J I' \ J', 

I \ J ^ I'/,]'. 

One consequence of this proposition is that a half-turn rotation of P reverses all four orders. We shall also 
use the fact that, if P' is obtained by applying a clockwise quarter-turn rotation to P, then I \ J <=> J' y V . 



3. A bijection between S-equivalence classes of floorplans and 
(2-14-3, 3-41-2)-avoiding permutations 

In this section we define S-equivalence of floorplans and construct a map S from floorplans to permutations. 
We show that S induces an injection from S-equivalence classes to permutations. We then characterize the 
class of permutations obtained from floorplans in terms of (generalized) patterns. 

3.1. S-equivalence 

Definition 3.1. Two floorplans Pi and P2 of size n + 1 arc S-equivalent if it is possible to label the segments 
of Pi by Ii, I2, ■ . ■ , I n and the segments of Pi by J\, J2, . . ■ , J n so that for all k,m e [n] , the segments Ik and 
I m exhibit the same neighborhood relation as Jr and J m . 
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Fig. QT] shows two S-equivalent floorplans: in both cases, the left-right neighborhood relations are 1 ■*- 4, 2 <- 
4, 3 <- 4, 4^5, 4 <- 6, and the below-above neighborhood relations are 2 J, 1, 3 | 2, 6 | 5. These floorplans 
are not R-equivalent, as can be seen by constructing their R-pcrmutations. We will prove in Section 0] that, 
conversely, R-equivalence implies S-equi valence. 
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Figure 11. Two S-equivalent (but not R-equivalent) floorplans. 



3.2. S-permutations 



5 bni 



Let P be a floorplan of size n + 1. There are n segments in P. Let S(P) be the sequence £11,62 
where bi is the label in the / order of the segment labeled i in the \ order, for all 1 < i < n. Then S(P) 
is a permutation of [n]; we call it the S- permutation of P and denote it by S(P). Equivalcntly, if ,I n 
is the list of segments in the \ order, then I a -iru, . . . ,I a -i( n \ is the list of segments in the / order, with 
a = S(P). Since the \- and ^-orders on segments can be determined in linear time (Proposition 12.11")) . the 
S-permutation is also constructed in linear time. An example is shown in Fig. 1121 

Thus, we assign a permutation to a floorplan in a way similar to that used in [I], but this time we use order 
relations between segments rather than rectangles. Note that S(P) is a permutation of [n], while R(P) is a 
permutation of [n + 1]. 

By definition of S(P), if a segment of P is labeled i in the \ order and j in the / order, then S(P)(i) = j. 
In other words, the graph of S(P) contains the point (i,j), which will be denoted by iVj. 
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S(P)=7 8 6 1 9 12 5 2 3 4 10 11 



Figure 12. A floorplan P, with segments labeled (i,j), where i (respectively, j) is the label 
according to the \ (respectively, / ) order, and the corresponding S-pcrmutation. 



It follows from Proposition 12. 121 that the map S has a simple behavior with respect to symmetries. 



ORDERS INDUCED BY SEGMENTS IN FLOORPLANS 



11 



Proposition 3.2. Let P be a (square) floorplan, and P' be obtained by reflecting P in the first diagonal. Let 
a = S(P) and a' = S(P'). Then a' is obtained by reading a from right to left or equivalently, by reflecting the 
graph of a in a vertical line. 

If instead P' is obtained by reflecting P in a horizontal line, then a' = er -1 . Equivalently, a' is obtained by 
reflecting the graph of a in the first diagonal. 

Proof. The following statements are equivalent: 

• cr(i) =j, 

• there exists a segment of P that has label i in the \ -order and j in the / -order (by definition of S), 

• there exists a segment of P' that has label n + 1 - i in the \ -order and j in the / -order (by 
Proposition |2T2|) . 

• a'(n + 1 - i) = j. 

This proves the first result. The proof of the second result is similar. □ 

Since the two reflections of Proposition l3. 21 generate the group of symmetries of the square, we can describe 
what happens for the other elements of this group: applying a rotation to P boils down to applying the same 
rotation to <S*(P), and reflecting P in A, a symmetry axis of the bounding square, boils down to reflecting 
S(P) in A', a line obtained by rotating A of 45° in counterclockwise direction. These properties will be 
extremely useful to decrease the number of cases we have to study in certain proofs. 

We will now prove that S(P) characterizes the S-equivalence class of P. Clearly, two S-equivalent floorplans 
give rise to the same orders, and thus to the same S-permutation. Conversely, let us define neighborhood 
relations between points in the graph of a permutation a as follows. Let Ni = (i,a(i)), Nj = (j, cr(j')) be 
two points in the graph of a. If i < j and a(i) < then the point Nj is to the NE of the point iVj. If, 

in addition, there is no i' such that i < i' < j and a(i) < o~(i') < o~(j), then Nj is a NE-neighbor of TV,. In a 
similar way we define when Nj is to the SE / SW / NW of Ni, and when the point Nj is a SE- / SW- / 
NW-neighbor of N t . For example, in the graph of Fig. [H the points (1,7), (2,8), (3,6), (5,9) and (6,12) 
are to the NW of TV? = (7,5); among them, (3,6), (5,9) and (6,12) are NW-neighbors of N?. 

The neighborhood relations between segments of P correspond to the neighborhood relations in the graph 
of S(P) in the following way. 

Observation 3.3. Let P be a floorplan, and let Ii and Ij be two segments in P. 

The segment Ij is to the right of Ii if and only if the point Nj lies to the NE of Ni. Consequently, Ij is a 
right-neighbor of Ii if and only if Nj is a NE-neighbor of Ni. 

Similar statements hold for the other directions: left (respectively, above, below) neighbors in segments 
correspond to SW- (respectively, NW-, SE-) neighbors in points. 

Proof. The segment Ij is to the right of Ii if and only if Ii / Ij and Ii \ Ij. By construction of a = S(P), 
this means that i < j and a(i) < o~(j). Equivalently, Nj lies to the NE of Ni. □ 

Remark. An analogous fact holds for rectangles of a floorplan and points in the graph of the corresponding 
R-permutation. It is not stated explicitly in pQ, but follows directly from the definitions in the same way as 
Observation 13.31 docs. 

Since the neighborhood relations characterize the S-cquivalence class, we have proved the following result. 

Corollary 3.4. Two floorplans are S-equivalent if and only if they have the same S-permutation. 

3.3. (2-14-3, 3-41-2)-Avoiding permutations 

In this section we first discuss the dash notation and bar notation for pattern avoidance in permutations, 
and then prove several facts about (2-14-3, 3-41-2)-avoiding permutations. As will be proved in Section I3~4l 
these are precisely the S-permutations obtained from floorplans. 
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In the classical notation, a permutation tt = a\a2 ■ ■ ■ a n avoids a permutation (a pattern) t = 6162 ■ • ■ bk if 
there are no 1 < i\ < 12 < ■ ■ ■ < ik < n such that a^a^ . . . ai k (a subpermutation of n) is order isomorphic to r 
{b x < by if and only if a lx <a iy ). 

The dash notation and the bar notation generalize the classical notation and provide a convenient way to 
define more classes of restricted permutations. For a recent survey, see |35j . 

In the dash notation, some letters corresponding to those from the pattern r may be required to be adjacent 
in the permutation 7r, in the following way. If there is a dash between two letters in r, the corresponding 
letters in 7r may occur at any distance from each other; if there is no dash, they must be adjacent in 7r. For 
example, 7r = a\ai ■ ■ ■ a n avoids 2-14-3 if there are no 1 < i < j < £ < m such that £ = j + 1 and aj < a, < a m < a^. 

In the bar notation, some letters of r may have bars. A permutation it avoids a barred pattern r if every 
occurrence of the unbarred part of r is a sub-occurrence of r (with bars removed). For example, 7r = 0102 . . . a n 
avoids 21354 if for any l<i<j<£<m such that aj < ai < a m < at, there exists k such that j < k < £ and 
ai < at- < a m (any occurrence of the pattern 2154 is a sub-occurrence of the pattern 21354). 

A (reduced) Baxter permutation is a permutation of [n] satisfying the following condition: 
There are no <,me [n\ satisfying i < j < £ < m, £ = j + 1, such that 
either 7r(j) < 7r(m) < 7r(i) < ir(£) and ir(i) = n(m) + 1, 
or n(£) < ir(i) < n(m) < Tr(j) and 7r(m) = ir(i) + 1. 

In the dash notation, Baxter permutations are those avoiding (2-41-3,3-14-2), and in the bar notation, 
Baxter permutations are those avoiding (41352,25314) (see [21] or [35) Sec. 7]). As proved in Q], the per- 
mutations that are obtained as R-permutations are precisely the Baxter permutations. It turns out that 
the permutations that are obtained as S-permutations may be defined by similar conditions, given below in 
Proposition 13.61 As in the Baxter case, these conditions can be defined in three different ways. 

Lemma 3.5. Let w be a permutation of[n]. The following conditions are equivalent: 

(1) There are no i,j,£,m e [n] such that i < j < £ < m, £ = j + tt(j) < 7r(*) < 7r(m) < 7r(£) 7 n(m) = 7r(i) + l. 

(2) In the dash notation, n avoids 2-14-3. 

(3) In the bar notation, tt avoids 21354. 

Fig. [13] illustrates these three conditions. The rows (respectively, columns) marked by dots in parts (1) 
and (2) denote adjacent rows (respectively, columns). The shaded area in part (3) does not contain points of 
the graph. 



© 



i j I m i j I m i j I m 



Figure 13. Three ways to define permutations avoiding 2-14-3. 



Proof. It is clear that 3 =>■ 2 => 1: the four points displayed in Fig. ITBTl) form an occurrence of the pattern 
of Fig. [T3T 2). and the four points displayed in Fig. [T3T2) form an occurrence of the pattern of Fig. 11373'). 
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Conversely, let us prove that if a permutation 7r contains the pattern 21354, then there exist i' ,j' ,£' ,m' as 
in the first condition. Assume that there are i < j < £ < m such that 7r(j) < < ir(m) < 7r(£), and there 
is no k such that j < k < I and 7r(i) < ir(k) < 7r(m). Let j' be the maximal number for which j < j' < I and 
ir(j') < 7r(i). Let £' = j' + 1. Then ir(i') > 7r(m), and we have a pattern 2-14-3 with i,j',£',m. 

Furthermore, let i' be the number satisfying i' < j' and 7r(i) < ir(i') < 7r(m), for which 7r(i') is the maximal 
possible. Let m! = n~ l (it(i') + 1). Then m! > £' and 7r(m') = + 1, and, thus, the first condition holds with 
i',j\e,m'. □ 

A similar result holds for permutations that avoid 3-41-2. Therefore, the following proposition holds. 

Proposition 3.6. Let a be a permutation of [n]. The following statements are equivalent: 

(1) There are no £, m e [n] satisfying i < j < £ < m, i = j + 1, such that 

either o~(j) < o~(i) < o~(m) < o~(£) and a(m) = o~(i) + 1, 
or a{t) < <j{m) < a(i) < a(j) and a(i) = a(m) + 1. 

(2) In the dash notation, a avoids 2-14-3 and 3-41-2. 

(3) In the bar notation, a avoids 21354 and 45312. 

Corollary 3.7. The group of symmetries of the square leaves invariant the set of (2-14-3, 3-41-2) -avoiding 
permutations. 

Proof. The second description in Proposition 13. 61 shows that the set of (2-14-3, 3-41-2)-avoiding permutations 
is closed under reading the permutations from right to left. The first (or third) description shows that it is 
invariant under taking inverses, and these two transformations generate the symmetries of the square. □ 

We shall also use the following fact. 

Lemma 3.8. Let a be a (2-14-3, 3-41-2) -avoiding permutation of \n\. Then no point in the graph of a has 
several NW-neighbors and several NE-neighbors. Similar statements hold for other pairs of adjacent diagonal 
directions. 

Proof. Assume that Ni = (i, cr(i)) has several NW-neighbors and several NE-neighbors. Let i' be the maximal 
number for which Ny is a NW- neighbor of Ni, and let Nj be another NW-neighbor of Ni. Then we have 
j < i' and a(i) < o~(j) < tr(i'). We conclude that i' = i - 1: otherwise a(i' + 1) < a(i) and, therefore, j, + l,i 
form the forbidden pattern 3-41-2, which is a contradiction. 

Similarly, if i" is the minimal number for which Ny is a NE- neighbor of Ni, then i" = i + 1. Let iVfc be 
another NE-neighbor of Ni. We have a(i) < a(k) < cr(i + 1). 

Assume without loss of generality that a(i - 1) < a(i + 1). Now, if cr(j) < cr(fc), then + l,k form the 
forbidden pattern 2-14-3; and if o~(k) < cr(j), then j,i - l,i,k form the forbidden pattern 3-41-2, which is, 
again, a contradiction. □ 

3.4. S-permutations coincide with (2-14-3, 3-41-2)-avoiding permutations 

By Corollary |3.41 the map S induces an injection from S-cquivalence classes of floorplans to permutations. 
Here, we characterize the image of S. 

Theorem 3.9. The map S induces a bisection between S-equivalence classes of floorplans of size n + 1 and 
(2-14-3, 3-41-2) -avoiding permutations of size n. 

The proof involves two steps: In Proposition 13 . 1 1 1 we prove that all S-permutations are (2-14-3, 3-41-2)- 
avoiding. Then, in Proposition 13.121 we show that for any (2-14-3, 3-41-2)-avoiding permutation a of [n], 
there exists a floorplan P with n segments such that S(P) = a. 

Recall that a horizontal segment has at most one left-neighbor and at most one right-neighbor, and a 
vertical segment has at most one below-neighbor and at most one above-neighbor. This translates as follows 
in terms of S-pcrmutations. 
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Observation 3.10. Let L be a segment in a floorplan P, and let N be the corresponding point in the graph 
of S(P). If Ii is a horizontal segment, then the point Ni has at most one NE-neighbor and at most one 
SW-neighbor. Similarly, if Ii is a vertical segment, then Ni has at most one SE-neighbor and at most one 
NW-neighbor. 

Proposition 3.11. Let P be a floorplan. Then S(P) avoids 2-14-3 and 3-41-2. 

Proof. By Proposition 13.21 the image of S is invariant by all symmetries of the square. Hence it suffices to 
prove that a = S(P) avoids 2-14-3. 

Assume that a contains 2-14-3. By Lemma [3~5l there exist i < j < £ < m, £ = j + 1 such that a(j) < a(i) < 
a(m) < a(£) and a(m) = a(i) + 1 (see Fig. [T4Tl)). We claim that the four segments Ii, Ij, Ie, I m are vertical. 

Consider Ij. The point Ni is a NE-neighbor of Nj. Consider the set {x ■ x > £, a(j) < u{x) < a{£)}. This 
set is not empty since it contains m. Let p be the smallest element in this set. Then N p is a NE-neighbor of 
Nj. Thus, Nj has at least two NE-ncighbors, Ni and N p . Therefore, Ij is vertical by Observation 13 . 1 Ol In a 
similar way one can show that Ii, Ii, I m are also vertical. 



© 



i j I m 



© Ij 



Figure 14. The pattern 2-14-3 never occurs in an S-permutation. 

By Observation [33] we have that: Ij J, L, I m { If, Ii <- It, Ij <- I m , Ii *- I m , Ij ■*- Ie- Moreover, the last two 
relations are neighborhood relations. Let Ik be the below- neighbor of Ii, and let Ik' be the below- neighbor of 
Ie (see Fig. [14] (2)). The segments Ik and Ik' are horizontal. If the line supporting Ik is (weakly) lower than 
the line supporting Ik>, then Ij (which is below /,) cannot be a left-neighbor of Ii since the interiors of their 
vertical projections do not intersect. Similarly, if the line supporting Ik is higher than the line supporting 
Ik', then Ii cannot be a left-neighbor of I m . We have thus reached a contradiction, and a cannot contain 
2-14-3. □ 

Proposition 3.12. For each (2-14-3, i-Al-2)- avoiding permutation a of \n\, there exists a floorplan P with 
n segments such that S(P) = a. 

Proof. We construct P on the graph of a. The boundary of the graph is also the boundary of P. For each 
point Ni = (i, cr(i)) of the graph, we draw a segment Ki passing through Ni according to certain rules. We first 
determine the direction of the segments Ki (Paragraph A below), and then the coordinates of their cndpoints 
(Paragraph B). We prove that wc indeed obtain a floorplan (Paragraph C), and that its S-pcrmutation is a 
(Paragraph D). This is probably one of the most involved proofs of the paper. 

A. Directions of the segments Ki. 

Let Ni = (i,a(i)) be a point in the graph of a. Our first two rules are forced by Observation 13.101 They are 
illustrated in Fig. [15] (no point of the graph lies in the shaded areas): 

• If Ni has several NW-neighbors or several SE-neighbors, then Ki is horizontal. 
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• If Ni has several SW-neighbors or several NE-nciglibors, then Ki is vertical. 
By Lemma 13.81 these two rules never apply simultaneously to the same point iVj. If one of them applies, we 
say that N is a strong point. Otherwise, Ni is a weak point. This means that N has at most one neighbor 
in each direction. 



© © © © 




Figure 15. Rules for determining the direction of the segment Ki passing through a strong point. 

We claim that if Ni and Nj are weak points, then they are in adjacent rows if and only if they are in 
adjacent columns. Due to symmetry, it suffices to show the if direction. Let Ni and N + i be weak points, 
and assume without loss of generality that a(i) < a{i + 1). If a{i + 1) - a(i) > 1, then there are points of the 
graph of a between the rows that contain Ni and iVj+i; thus, either Ni has at least two NE-neighbors or N+i 
has at least two SW-ncighbors, which means that one of them at least is strong. Hence a(i + 1) = a(i) + 1. 

Thus, weak points appear as ascending or descending sequences of adjacent neighbors: Ni,N + i, . . . ,N+£ 
with a(i) = a(i + 1) - 1 = ••• = a(i + £) - 1 or a(i) = a(i + 1) + 1 = ••• = a{i + £)+£. Note that a weak point Ni 
can be isolated. 

For weak points, the direction of the corresponding segments is determined as follows: 

• If Ni, Ni+i, . . . , N + £ is a maximal ascending sequence of weak points, then the directions of Ki, Ki + \, . . . , 
Ki + i are chosen in such a way that Kj and Kj+i are never both horizontal, for i < j < i + 1. Hence 
several choices are possible (this multiplicity of choices is consistent with the fact that all S-equivalent 
floorplans give the same permutation). 

• If Ni, N+i, . . . , Ni + £ is a maximal descending sequence of weak points, then the directions oi Ki,Ki + \, . . . , 
Ki+i are chosen in such a way that Kj and Kj + i are never both vertical, for i < j < i + £. 

In particular, for an isolated weak point Ni, the direction of Ki can be chosen arbitrarily. 
B. Endpoints of the segments Ki. 

Once the directions of all Ki's arc chosen, their endpoints are set as follows (see Fig. [16] for an illustration): 

• If Ki is vertical (which implies that N has at most one NW- neighbor and at most one SE- neighbor): 

— If Ni has a NW-neighbor Nj, then the upper endpoint of Ki is set to be (i,a(j)). We say that 
Nj bounds Ki from above. Otherwise (if JVj has no NW-neighbor), Ki reaches the upper side of 
the boundary. 

— If N has a SE-neighbor Nk, then the lower endpoint of Ki is (i,a(k)). We say that Nk bounds 
Ki from below. Otherwise, Ki reaches the lower side of the boundary. 

• If Ki is horizontal (which implies that N has at most one SW- neighbor and at most one NE- neighbor) : 

— If N has a SW- neighbor Nj, then the left endpoint of Ki is (j,a(i)). We say that Nj bounds 
Ki from the left. Otherwise, Ki reaches the left side of the boundary. 

— If N has a NE-neighbor Nk, then the right endpoint of Ki is (k,a(i)). We say that Nk bounds 
Ki from the right. Otherwise, Ki reaches the right side of the boundary. 
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j i k j i k 

Figure 16. Determining the endpoints of the segment Kf. the points Nj and Nk bound the 
segment Ki. 



Fig. [T7] presents an example of the whole construction: in Part 1, the directions are determined for strong 
(black) points, and chosen for weak (gray) points; in Part 2, the endpoints are determined and a floorplan is 
obtained. Notice that a is the S-permutation associated with the floorplan P of Fig. [TJl but here we have 
obtained a different floorplan, P' . We leave it to the reader to check that another choice of directions of 
segments passing through weak points leads to P. 

The question of when S(P) = S(P') will be studied in Section l4~2l 
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Figure 17. Constructing a floorplan from a (2-14-3, 3-41-2)-avoiding permutation. 

Remark. Using dynamic programming, one can determine in linear time the values 

rrii = max{fc < i : a(k) < a(i)} u {0} and M, = max{/c < i : cr(fc) > cr(i)} u {0}. 

By applying this procedure to a and to the permutations obtained by applying to a a symmetry of the square, 
one can decide in linear time, for each point Ni of <r, if it has one or several NW-neighbours and locate one 
of them. This implies that the above construction of a floorplan starting from a (2-14-3, 3-41-2)-avoiding 
permutation can be done in linear time. 

C. The construction indeed determines a floorplan. 

In order to prove this, we need to show that two segments never cross, and that the endpoints of any segment 
Ki are contained in segments perpendicular to Ki (unless they lie on the boundary). The following observation 
will simplify some of our proofs. 

Observation 3.13. Let a be a (2-14-3, 3-41-2) -avoiding permutation, and let a' be obtained by applying 
a rotation p (a quarter turn or a half-turn, clockwise or counterclockwise) to (the graph of) a. If P is a 



ORDERS INDUCED BY SEGMENTS IN FLOORPLANS 



17 



configuration of segments obtained from a by applying the rules of Paragraphs A and B above, then p(P) can 
be obtained from a' using those rules. 

To prove this, it suffices to check that the rules are invariant by a 90° rotation, which is immediat^E 

C.l. Let Ki be a vertical (respectively, horizontal) segment, and let Nj and be the points 
that bound it. Then the segments Kj and are horizontal (respectively, vertical). 

Thanks to Obscrvation l3.l5I it suffices to prove this claim for a vertical segment Ki and for the point Nj that 
bounds it from above. We need to prove that Kj is a horizontal segment. 

We have j < i and a(i) < <r(j), and, since Nj is a NW-neighbor of N, there is no £ such that j < £ < i 
and a(i) < a(£) < cr(j). Furthermore, there is no t such that j < £ < i, a(j) < a(£), or such that £ < j, 
a(i) < a{£) < a(j): otherwise N would have several NW-neighbors and, therefore, Ki would be horizontal. 
Now, if i - j > 1, then there exists £ such that j < £ < i, a{£) < <r(i); and if <r(j) - a(i) > 1, then there exists 
m such that i < m, a(i) < <r(m) < o~(j). In both cases Nj has several SE-neighbors, and, therefore, Kj is 
horizontal as claimed. 

It remains to consider the case where j = i - 1 and a(j) = a(i) + 1. If the point N is strong, then (since 
Ki is vertical) it has several NE-neighbors or several SW-neighbors. Assume without loss of generality that 
N has several NE-neighbors. Let £ be the minimal number such that N( is a NE-neighbor of N, and let 
N m be another NE-neighbor of JV». Then we have a(i - 1) < <r(m) < a(£) and a(£ - 1) < o'(i). However, then 
i - 1,£ - l,£,m form a forbidden pattern 2-14-3. Therefore, N is a weak point. Clearly, N-i as a unique 
SE-neighbor (which is N). Its NE- and SW-neighbors coincide with those of N, so that there is at most 
one of each type. Thus if iVi_i is strong, it has several NW-neighbors, and Ki-i is horizontal, as claimed. If 
Ni-i is weak, then the rules that determine the direction of the segments passing through (descending) weak 
points implies that K^\ and Ki cannot be both vertical. Therefore, Kj = AVi is horizontal, as claimed. 

C.2. If Nj and Nk are the points that bound the segment Ki, then the segments Kj and Kk 
contain the endpoints of Ki 

Thanks to Observation 13. 13l it suffices to show that if Ki is a vertical segment and Nj bounds it from above, 
then Kj (which is horizontal as shown in Paragraph C.l above) contains the point (i,o~(j)). We saw in 
Paragraph C.l that in this situation there is no £ such that j < £ < i, a(j) < a(£). This means that there is 
no point Ni that could bound Kj from the right before it reaches (i,a(j)). 

C.3. Two segments Ki and Kj cannot cross 

Assume that Ki and Kj cross. Assume without loss of generality that Ki is vertical and Kj is horizontal, so 
that their crossing point is (i,a(j)). We have either i < j or j < i, and a(i) < o~(j) or cr(j') < o~(i). Assume 
without loss of generality j < i and a(i) < cr(j). Then Nj is to the NW of N. The ordinate of the (unique) 
NE-neighbor of N is hence at most cr(j'). By construction, the upper point of Ki has ordinate at most u(j), 
while Kj lies at ordinate o~(j), and thus Ki and Kj cannot cross. 

We have thus proved that our construction indeed gives a floorplan. Let us finish with an observation on 
joins of segments of this floorplan, which follows from Paragraph C.2 and is illustrated below. 

Observation 3.14. Suppose that a vertical segment Ki and a horizontal segment Kj join at the point 
(i,<r(j))- Then: 

• If the join of Ki and Kj is of the type T, then i > j. 

• If the join of Ki and Kj is of the type 1, then i < j . 

• If the join of Ki and Kj is of the type h, then a(i) < o~(j). 

• If the join of Ki and Kj is of the type -f, then a(i) > o~(j). 



^That the construction has the other symmetries of Proposition 13.21 is also true, but less obvious. We shall only use Obser- 
vation 13.131 
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D. For any floorplan P obtained by the construction described above, S(P) = a. 

This (concluding) part of the proof is given in Appendix |XJ □ 

4. Relations between the R- and S-permutations 

In this section we prove that if two floorplans are R-equivalent, they are S-equivalent (that is, the In- 
equivalence refines the S-equi valence) . In fact, we give a simple graphical way to construct S(P) from R(P), 
which also shows that S(P) and R(P) taken together form the complete Baxter permutation associated with 
the reduced Baxter permutation R(P). Finally, we characterize the R-equivalencc classes that belong to the 
same S-cqui valence class. 

4.1. Constructing S(P) from R(P) 

Let P be a floorplan of size n + 1. We draw the graphs of p = R(P) and a = S(P) on the same diagram in 
the following way (Fig. I18j) . For the graph of p we use an(n+l)x(n + l) square whose columns and rows 
are numbered by 1,2, . . . ,n + 1. The points of the graph of p are placed at the centers of these squares, and 
these points are black. For the graph of a we use the grid lines of the same drawing, when the ith vertical 
(respectively, horizontal) line is the grid line between the ith and the (i + l)st columns (respectively, rows). 
The point (i,o~(i)) is placed at the intersection of the ith vertical grid line and the jth horizontal grid line, 
where j = o~(i). Such points are white. The whole drawing is called the combined diagram of P. Note that 
the extreme (rightmost, leftmost, etc.) grid lines are not used. 



(1,8) (6,13) 
(3,9) 

(2, 7 ) (12,12) 

1 (7, 10) 

(5,6) 



(4, 1) 



(8,3) 



(9, 2) 



(10, 5) 



(11,4) 



(13, 11) 



(2,8) 



(1,7) 



(3,6) 



(5, 9) 



(6, 12) 



(11, 10) 



(7, 5) 



(4, 1) 



(9, 3) 



(8, 2) 



(10, 4) 



(12, 11) 





13 


12 






12 


11 






11 


10 






10 


9 




8 


9 


7 


8 




7 


6 






6 


5 






5 


4 






4 


3 






3 


2 






2 







• 
















j 






• 

-c 


> 
• 












C 










m 






> 






• : 




j 












-■-■:-■-< 
• 


> - 
















■•-0-- 
• 


- o - 




















• 






















c 


: • 


















;•■•< 


J- 
• 
















: O- - 

O '■- - 


















• 










: • 















1 2 3 4 5 6 7 8 9 10 11 12 13 
1 2 3 4 5 6 7 8 9 10 11 12 



(T) Rectangles 



Segments 



@ Combined diagram 



Figure 18. The floorplan P from Fig. [T2j (1) The labeling of rectangles; (2) The labeling 
of segments; (3) The combined diagram: R(P) = 87916 13 10 3254 12 11 (black points) 
together with S(P) = 78619 12 5234 10 11 (white points). 



Definition 4.1. Let p be a Baxter permutation of [n + 1]. For 1 < i < n, define ji as follows: 

• if p(i) < p(i + 1), then ji = max{p(fc), k < i and p(k) < p(i + 1)}, 

• if p(i) > p(i + 1), then ji = max{p(fc), k > i + 1 and p(k) < p(i)}. 
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The definition of Baxter permutations implies that 

• if p(i) < p(i + 1), k > i + 1 and p(k) > p(i), then p(k) > p(ji), 

• if p(i) > p(i + 1), k < i and p(k) > p(i + 1), then p(k) > p(ji). 

Theorem 4.2. Let P be a floorplan of size n + 1, and let p = R(P). Then S(P) = (ji, J2, • ■ • ,jn)> where ji is 
defined in Definition ^. 1\ In particular, R- equivalent floorplans are also S-equivalent. 

Returning to the original papers on Baxter permutations (see for instance [111 Thm. 2], or the definition 
of complete permutations in [12[ p. 180]) this means that the combined diagram forms a (complete) Baxter 
permutation tt. The points of R(P) form the reduced Baxter permutation ir associated with n, and the 
points of S(P) are those that are deleted from 7r when constructing ir . 

Proof. Let i e [n]. Denote a = S(P) and j = a(i). Then the segment Ii labeled i in the \ order, is labeled j 
in the ^ order. We denote by Aj- (resp. B k ) the fcth rectangle in the \- (resp. / -) order. We wish to prove 
that j=ji. 

Assume first that Ii is horizontal. By Observation 12.111 the rightmost rectangle whose lower side is 
contained in Jj is Ai, and the leftmost rectangle whose upper side is contained in 7j is A i+1 (Fig. H9| . 

By definition of p, we have A^ = B p ^ for all k. By symmetry, since Ii is the jth segment in the / order, 
the rightmost rectangle whose upper side is contained in Ii is B-i , and the leftmost rectangle whose lower side 
is contained in I { is B j+1 . There holds A i+1 / B j / B J+1 / A { and B ]+1 \ A, \ A i+1 \ BK By definition 
of p = R(P), this means p(i + l)<j<j + l< p(i) and p _1 (j + l)<i<i + l< p _1 (j). This shows that j 
coincides with the value ji of Definition 14.11 (for the case p(i) > p(i + 1)). 

The case where Ii is vertical is similar, and corresponds to an ascent in p. □ 



B j+l 


h 




A- , 




B j 



Figure 19. Illustration of the proof of Theorem 14.21 

The symmetry in the definition of j, makes the following property obvious, without going through floor- 
plans. 

Corollary 4.3. Let P be a floorplan and let p = R(P) be the corresponding Baxter permutation. Let us abuse 
notation by denoting S(p) ■= S(P). If p' is obtained by applying to p a symmetry of the square, then the same 
symmetry, applied to S(p), gives S(p'). 

Remark. The combined diagram is actually the R-permutation of a floorplan of size 2n + 1. Indeed, let P 
be a floorplan of size n + 1. If we inflate segments of P into narrow rectangles, we obtain a new floorplan of 
size 2n + 1, which we denote by P (Fig. [2~CT|) . Observe that a rectangle of P corresponding to a rectangle A of 
P has a unique above (respectively, right, below, left) neighbor, which corresponds to the segment of P that 
contains the above (respectively, right, below, left) side of A. 

It follows from Observation 12.111 and Fig. [9] that the \ order in P is A1I1A2I2 ■ ■ ■ A n I n A n+ \ . It is thus 
obtained by shuffling the \ orders for rectangles and segments of P. Symmetrically, the / order in P 
is A p -i(^\ J .-i( 1 )"-j4p-i(„)J cr -i( T[ )j4p-i( n+1 ). Thus the combined diagram of R(P) and S(P), as in Fig. [T51 
coincides with the graph of R(P). 
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Figure 20. Inflating the segments of a floorplan. 



4.2. Floorplans that produce the same S-permutation 

In this section we characterize in terms of their R-permutations the floorplans that have the same S- 
pcrmutation. This will play a central role in the enumeration of S-pcrmutations. 

We first describe the floorplans whose S-pcrmutation is 123 . . . n. Such floorplans will be called ascending 
F-blocksB It is easy to see that in an ascending F-block, all vertical segments extend from the lower to the 
upper side of the boundary, and there is at most one horizontal segment between a pair of adjacent vertical 
segments (this can be shown inductively, by noticing that at most one horizontal segment starts from the 
left side of the bounding rectangle). See Fig. EH Conversely, every floorplan of this type has S-permutation 
123... n. Therefore, an ascending F-block consists of several rectangles that extend from the lower to the 
upper side of the boundary, some of them being split into two sub-rectangles by a horizontal segment. The 
corresponding R-permutations are those that satisfy \p(i) —i\ < 1 for all 1 < i < n+1. The number of ascending 
F-blocks of size n+1 (and, therefore, the number of such permutations) is the Fibonacci number F n+ i (where 
F = F 1 = l). 




Figure 21. The 8 ascending F-blocks for n = 4, and their R-permutations. 

A similar observation holds for the floorplans whose S-permutation is n ... 321. Such floorplans are called 
descending F-blocks. In descending F-blocks, all horizontal segments extend from the left side to the right side 
of the boundary, and there is at most one vertical segment between a pair of adjacent horizontal segments. 
In other words, descending F-blocks consist of several rectangles that extend from the left to the right side 
of the boundary, some of them being split into two sub- rectangles by a vertical segment. The corresponding 
R-permutations are characterized by the condition \p(i) — (n + 2 - i)\ < 1 for all 1 < i < n + 1. 

For an F-block F, the size of F (that is, the number of rectangles) will be denoted by If \F\ = 1, we 
say that F is a trivial F-block. Note that if \F\ < 2, then F is both ascending and descending, while if |F| > 3, 
then its type (ascending or descending) is uniquely determined. 



^The letter F refers to Fibonacci, for reasons that will be explained further down. 
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Let P be a floorplan. We define an F-block in P as a set of rectangles of P whose union is an F-block, as 
defined above. In other words, their union is a rectangle, and the S-permutation of the induced subpartition 
is either 123... or ...321. The F-blocks of P are partially ordered by inclusion. Since segments of P do 
not cross, a rectangle in P belongs precisely to one maximal F-block (which may be of size 1). So there is a 
uniquely determined partition of P into maximal F-blocks (Fig. [22] left). 

A block in a permutation p is an interval such that the values {/?(«), ■ • ■ , p(j)} also form an interval [3]. 
By extension, we also call a block the corresponding set of points in the graph of p. Consider £ rectangles in P 
that form an ascending (respectively, descending) F-block. By Observation ^. Ill and the analogous statement 
for the ,/ order, these £ rectangles form an interval in the X and / orders. Hence the corresponding £ points 
of the graph of R(P) form a block, and their inner order is isomorphic to a permutation r of [£] that satisfies 
|r(i) - i\ < 1 (respectively, \r(i) - (£ + 1 - i)\ < 1) for all 1 < i < I. 

The converse is also true: If £ points of the graph of R(P) form an £ x £ block, and their inner order 
is isomorphic to a permutation r of [£] that satisfies |r(i) - i\ < 1 (respectively, \r(i) - (£ + 1 - i)\ < 1) for 
all 1 < i < £, then the corresponding rectangles in P form an ascending (respectively, descending) F-block. 
Indeed, let H be such an ascending block in the graph of R(P). Let us partition the points of H in singletons 
(formed of points that lie on the diagonal) and pairs (formed of transposed points at adjacent positions). Let 
Ql, Q2, ... be the parts of this partition, read from the SW to the NE corner of H. For each i = 1,2,..., the 
point(s) of Qi+\ are the only NE-neighbors of the point(s) of Qi, and, conversely, the point(s) of Qi are the 
only SW-neighbors of the point(s) of Qi+\. Therefore, by the remark that follows Observation 13. 3[ the left 
side of the rectangle(s) corresponding to the point(s) of Qi+i coincides with the right side of the rectangle(s) 
corresponding to the point (s) of Qi. If Qi consists of two points then we have two rectangles whose union is 
a rectangle split by a horizontal segment. The argument is similar for a descending block. 

Therefore, such blocks in the graph of p will be also called ascending (respectively, descending) F-blocks. 
Fig. 1221 shows a floorplan with maximal F-blocks denoted by bold lines, and the F-blocks in the corresponding 
permutation R(P) (the graph of S(P) is also shown). 
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Figure 22. Maximal F-blocks in floorplans and in permutations. 

Let Fi,Fz,... be all the maximal F-blocks in the graph of p (ordered from left to right). For i > 1, let 
[j/i, y'j] be the interval of values p(j) occurring in Fi, and define di ■= + if Fi is ascending, and di •■= - if Fi is 
descending (di is left undefined if Fi has size 1 or 2). The F-structure of p is the sequence .Fi, F2, . . . , where 
Fi = ([yiiVilidi). For example, the F-structurc of the permutation in Fig. |2"21 is 

([7,9],+), ([!]), ([6]), ([13]), ([10]), ([2,5],+), ([11,12]). 
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Theorem 4.4. Let P\ and P2 be two floorplans with n segments. Then S(Pi) 
and R(P2) have the same F- structure. 



S(P 2 ) if and only if R(P%) 



In other words, S(Pi) = S(p2) if and only if R(Pi) and R{P2) may be obtained from each other by 
replacing some F-blocks F\,F2,.-. with, respectively, F-blocks F^F^,..., where Fi is S-equivalent to F[ for 
all i. 

Proof. The "if" direction is easy to prove. Assume R{P\) and RiP-i) have the same F-structure. In view of 
the way one obtains S(P) from R(P) (Theorem 14. 2[) . we have S(Pi) = S(P2). Observe in particular that 
inside a maximal F-block of R(P), the points of S(P) are organized on the diagonal (in the ascending case) 
or the anti-diagonal (in the descending case). 

In order to prove the "only if" direction, we will first relate, for a point of S(P), the fact of being inside 
a maximal F-block to the property of being weak. (Recall that a point AT* in the graph of S(P) is weak if it 
has at most one neighbor in each of the directions NW, NE, SE, SW, and strong otherwise.) If a maximal 
F-block of R(P) occupies the area [x,x'] x [y,y'], then the point AT* = is inside this block if x < i < x' 
and y < j < y' . For example, in Fig. [22] six points in the graph of S(P) (the white points in the combined 
diagram) are inside a maximal F-block: (1,7), (2,8), (8,2), (9,3), (10,4) and (12,11). Observe that the 
notion of "being inside" a maximal F-block is a priori relative to R(P). However, the following proposition 
shows that it is an intrinsic notion, depending on S(P) only. 

Lemma 4.5. Let TV, be a point in the graph of a = S(P). Then Ni is inside a maximal F-block of R(P) if 
and only if it is a weak point of S(P) . 

This lemma is proved in Appendix [B] and the rest of the theorem in Appendix [C] □ 
5. Enumeration of (2-14-3, 3-41-2)-avoiding permutations 



It follows from Theorem |L4] that S-permutations of size n are in bijection with Baxter permutations of size 
n + 1 in which all maximal ascending F-blocks are increasing (that is, order isomorphic to a permutation of the 
form 123 . . . to), and all maximal descending F-blocks of size at least 3 are decreasing. A Baxter permutation 
that does not satisfy these conditions has at least one improper pair. 

Definition 5.1. Let p be a Baxter permutation. Two points of the diagram of p that lie in adjacent rows 
and columns form an improper pair if they form a descent in a maximal ascending F-block, or an ascent in a 
maximal descending F-block of size at least 3. 



This definition is illustrated in Fig. 



Observe that a point belongs to at most one improper pair. In 



particular, a permutation of size n + 1 has at most L 2 ! - ] improper pairs. 
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Figure 23. Improper pairs in maximal F-blocks. 
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Proposition 5.2. Let 

b _ A 2 /n + l\/n + l\/n + l\ 

fee t/ie number of Baxter permutations of size n (see |15| ). The number a n of (2-14-3, 3-41-2) -avoiding per- 
mutations of size n is 

L(»+i)/2j .(n + l-i\ 

a n = XI ( _1 M \K+l 
i=0 ' 



Proof. We have just explained that (2-14-3, 3-41-2)-avoiding permutations of size n are in bijection with 
Baxter permutations of size n + 1 having no improper pair. By the inclusion-exclusion principle, 

i>0 

where is the number of Baxter permutations of size n + 1 with i marked improper pairs. Let p be such 

a permutation, and contract every marked improper pair into a single (marked) point: this gives a Baxter 
permutation p' of size n + 1 - i, with i marked points. 

Observe that if two points of p are in the same maximal F-block, then their images, after contraction, are 
in the same maximal F-block of p' . (The converse is false: the permutation 1342 has two maximal F-blocks 
and one improper pair (consisting of the values 34). By contracting this pair, one obtains the permutation 
132, which is an F-block.) 

We claim that each Baxter permutation of size n+l — i with i marked points is obtained exactly once in our 
construction, and that the unique way to expand each marked point into an improper pair is the following: 

• if the marked point lies on the diagonal of an ascending maximal F-block (of size > 1), replace it by 
a descending pair of adjacent points, 

• if the marked point lies on the anti-diagonal of a descending maximal F-block of size > 2, replace it 
by an ascending pair of adjacent points, 

• otherwise, observe that the block has size at least 3; if it is ascending (rcsp. descending), and the 
marked point does not lie on the diagonal (resp. anti-diagonal), replace it by an ascending (resp. 
descending) pair of adjacent points. 

Details are left to the reader. 

This construction implies that the number of Baxter permutations of size n + 1 having i marked improper 
pairs is b n +i.i = ( n+1 ~ 4 )i> n+ i-j, and the proposition follows. □ 

Remarks 

1. Let A(t) be the generating function of (2-14-3, 3-41-2)-avoiding permutations, and let B(t) be the gener- 
ating function of (non-empty) Baxter permutations. The above result can be rewritten as 

A(t) = £i fc (i-0 fc+1 & fc+ i = 7B(i(l-0)- (1) 

Observe that t(l - 1) = s if t = sC(s), where C(s) = 1 ~ v ^~ 4s is the generating function of Catalan numbers. 
Hence, conversely, 

B(s) =sC(s)A(sC(s)). (2) 
This suggests that another connection between S- and R-permutations, involving Catalan numbers, exists. 

2. The form of a n and b n implies that A(t) and B(t) are D-finite, that is, satisfy a linear differential equation 
with polynomial coefficients [251 [57]. In fact, 

-12* + 6(1 - 2t)B{t) - 2t (-3 + Ut + 8t 2 ) B'(t) -t 2 (t+ 1) (St - 1) B"{t) = 
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and 

12 (t - 1) (2t - if + (104* - 338* 2 + 512t 3 - 294t 4 - 110t 5 + 192* 6 - 48t 7 - 12) A(t) 

-2t (t - 1) (40* 6 - 128t 5 + 89i 4 + 53t 3 - 88i 2 + 35* - 4) A'(t)-t 2 (2t - 1) (8* 2 - 8t + l) (t 2 - t - l) (t - if A"(t) = 0. 

This implies that the asymptotic behavior of the numbers a n and b n can be determined almost automatically 
(see for instance [T8] Sec. VII.9]). For Baxter permutations, it is known [32] that b n ~ 8 n n~ 4 (up to a multi- 
plicative constant, which can be determined thanks to standard techniques for the asymptotics of sums [29]). 
For On, we find a n ~ (4 + 2\/2) n n~ 4 . 

3. For 1 < n < 30, the number of (2-14-3, 3-41-2)-avoiding permutations of [n] is given in the following table, 
which we have sent to the OEIS [301 A214358]. 
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27277772831911348 


209275178482957838142 



6. The case of guillotine floorplans 

In this section we study the restriction of the map S to an important family of fioorplan called guillotine 
floorplans [H[22l[36]. 



Definition 6.1. A fioorplan P is a guillotine fioorplan (also called slicing fioorplan [21]) if either it consists 
of just one rectangle, or there is a segment in P that extends from one side of the boundary to the opposite 
side, and splits P into two sub-floorplans that are also guillotine. 

The restriction of the map R to guillotine floorplans induces a bijection between R-equivalence classes of 
guillotine floorplans and separable permutations (defined below) [I]. Here, we first characterize permutations 
that are obtained as S-permutations of guillotine floorplans, and then enumerate them. 

6.1. Guillotine floorplans and separable- by-point permutations 

A nonempty permutation a is separable if it has size 1, or its graph can be split into two nonempty blocks 
Hi and Hi, which are themselves separable. Then, either all the points in Hi are to the SW of all the points 
of H2 (then cr, as a separable permutation, has an ascending structure), or all the points in Hi are to the 
NW of all the points of Hi (then <r, as a separable permutation, has a descending structure). Separable 
permutations are known to coincide with (2-4-1-3, 3-l-4-2)-avoiding permutations |10| . In particular, they 
form a subclass of Baxter permutations. The number g n of separable permutations of [n] is the (n - l)st 
Schroder number [30( A006318], and the associated generating function is: 

C(t):=Z g n*"= 1 -'-^- 6f + ' 2 . (3) 
n>l Z 

Definition 6.2. A permutation a of [n\ is separable-by-point if it is empty, or its graph can be split into 
three blocks Hi, Hi, H3 such that 

Hi consists of one point N, 

- Hi and H3 arc themselves separable-by-point (thus, they may be empty), and 

- either all the points of Hi are to the SW of N, and all the points of H3 are to the NE of N (then a 
has an ascending structure), or all the points of Hi are to the NW of TV and all the points of H3 are 
to the SE of N (then a has a descending structure). 
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The letter N for the central block refers to the fact that we have denoted by Ni the point (i,a(i)) of an 
S-permutation a. Observe also that N necessarily corresponds to a fixed point of a if a is ascending, and to 
a point such that a(i) = n + I - i is a is descending and has size n. 




FIGURE 24. Separable-by-point permutations. 



See Fig. [53] for a schematic description and an example of separable- by-point permutations. For n < 3, all 
permutations are separable- by-point. It is clear that if a nonempty permutation a is separablc-by-point, then 
it is separable. The permutations 2143 and 3412 are separable, but not separablc-by-point. The following 
result characterizes separable-by-point permutations in terms of forbidden patterns. In particular, it implies 
that these permutations are S-permutations. 

Proposition 6.3. Let a be a permutation of \n\. Then a is separable-by-point if and only if it is 
(2-14-3, 3-41-2, 2-4-1-3, 3-1-4-2) -avoiding. 

Proof. Assume that a is separable- by-point. In particular, a is separable, and, therefore, it avoids 2-4-1-3 
and 3-1-4-2. Assume for the sake of contradiction that a contains an occurrence of 2-14-3, corresponding 
to the points Ni,Nj,Nj+i and Nk, and has a minimal size for this property. Then the points forming the 
pattern must be spread in at least two of the three blocks. This forces a to have an ascending structure, with 
Ni and Nj in one block, Nj+i and Nk in the following one (because Nj and Nj+i are adjacent). But this is 
impossible as the central block of a contains a unique point. Similarly one shows that a avoids 3-41-2. 

Conversely, we argue by induction on the size of a. Let a be a (2-14-3, 3-41-2, 2-4-1-3, 3-l-4-2)-avoiding 
permutation of [n\. For n < 3 there is nothing to prove. Let n > 4. Since a is (2-4-1-3, 3-l-4-2)-avoiding, 
it is separable. Assume without loss of generality that a (as a separable permutation) has an ascending 
structure: the first block is [1, i] x [1, i], the second block is [i + 1, n] x [i + 1, n] where 1 < i < n. If a(i) + i and 
cr(i + 1) + i + 1, then a (i),i,i + l,cr (i + 1) form a forbidden pattern 2-14-3. Thus, cr(i) = i or <r(i + 1) =i + l, 
and one obtains a three-block decomposition of a by choosing for the central block TV one of these two fixed 
points. The remaining two blocks avoid all four patterns, and, therefore are separablc-by-point themselves 
by the induction hypothesis. It follows that a is scparable-by-point. □ 

Theorem 6.4. A floorplan P is a guillotine floorplan if and only if S(P) is separable-by-point. 

Proof. Let P be a guillotine floorplan. We argue by induction on the size of P. If P consists of a single 
rectangle, then S(P) is the empty permutation, and is separable-by-point. Otherwise, consider a segment 
that splits P into two rectangles. Assume that this segment is U (that is, the ith. segment in the \ order) 
and that it is vertical. All the segments to the left (respectively, right) of U come before (respectively, after) 
Ii in the \ and / orders. Consequently: 

- Ii is also the ith segment in the / order, so that Ni = (i,i), 

- by Observation I3.3[ all the points of the graph of a that correspond to segments located to the left 
(respectively, right) of I\ arc to the SW (respectively, NE) of Ni. 
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Thus, we have three blocks Hi, H2 and i?3with an ascending structure. The blocks Hi and H3 are the 
S-permutations of the two parts of P, which are themselves guillotine: by the induction hypothesis, Hi and 
H3 are separable-by-point. Thus S(P) is separable-by-point with an ascending structure. 

Similarly, if Ii is horizontal, we obtain a separablc-by-point permutation with a descending structure. 

Conversely, assume that a := S(P) is separablc-by-point. We will prove by induction on n that P is a 
guillotine floorplan. 

The claim is clear for n = 1. For n > 1, assume without loss of generality that a has an ascending structure. 
Let H2 = be the second block in a decomposition of a. Then for all j < i, we have Ij <- ij, and for all 

j > i, we have ii *- Ij. Therefore, if ij is vertical, it has no below- or above-neighbors, and thus extends from 
the lower to the upper side of the boundary. The two sub-floorplans of P correspond respectively to the blocks 
Hi and H3: hence they are guillotine by the induction hypothesis. Suppose now that ij is horizontal. Then 
we have <r(i - 1) = i - 1 (if i > 1) and a(i + 1) = i + 1 (if i < n), since otherwise Ii has several left-neighbors or 
several right- neighbors (Observation [373]), which is never the case for a horizontal segment. Assume without 
loss of generality that i > 1. Then another block decomposition of a is obtained with the central block 
H' 2 = {(i- 1, 1)}, corresponding to the vertical segment Ii-i- The previous argument then shows that P is 
guillotine. □ 

6.2. Enumeration 

In this section we enumerate S-equivalence classes of guillotine floorplans, or equivalently, separable-by- 
point permutations. 

Proposition 6.5. For n > 1, let g n be the number of separable permutations of size n, and let G(t) the 
associated generating function, given by ([3]). The number h n of separable-by-point permutations of size n is 

L(n+1)/2J 

\9n+\-i- 



to V % J 



Equivalently, the generating function of separable-by-point permutations is 



H(t) = £ M n = e * n (i - tr v+i = 7<w - o) = 1 ~ t+ * 2 ~ Vl " 6 / +7 * 2 ~ 2 * 3+ * 4 . 

Proof. Recall that the R-permutations associated with guillotine floorplans are the separable permutations, 
and return to the proof of Proposition [ITU The contraction/expansion of points used in this proof preserves 
separability, so that we can apply the same argument, which yields directly the proposition. □ 

Remarks 

1. The first values are 1,2,6,20,70,254,948,3618,14058,55432. This sequence [IB A078482] also enumerates 
(2-4-3-1, 3-2-4-1, 2-4-1-3, 3-l-4-2)-avoiding permutations (or permutations sortable by a stack of queues), as 
found by Atkinson and Stitt Thm. 17]. 

2. Using the transfer theorems from [181 Sec. VI. 4], we can find the asymptotic behavior of the numbers h n : 

h ~l ? \ n -m 



\i- VW2-11, 
up to a multiplicative constant. 

3. A generalization of Proposition 16 . 51 to d-dimensional guillotine partitions is presented in [4]. 

7. Final remarks 

We have shown that many analogies exist between R- and S-equivalence. However, there also seems to be 
one important difference. Looking at Fig. [1] suggests that one can transform a floorplan into an R-equivalent 
one by some continuous deformation. In other words, R-equivalence classes appear as geometric planar 
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objects. This is confirmed by the papers [9| \17 \ [20j. which show that bipolar orientations of planar maps 
provide a convenient geometric description of R-equivalence classes of floorplans. However, S-equivalence is 
a coarser relation, and two S-equivalent floorplans may look rather different (Figs. HT1 and |2"T|) . It would be 
interesting to find a class of geometric objects that captures the notion of S-equivalence classes, as bipolar 
orientations do for R-cquivalcnce classes. 

In Section [5] we have established a simple enumerativc connection, involving Catalan numbers, between 
Baxter permutations and (2-14-3, 3, 41-2)-avoiding permutations. Is there a direct combinatorial proof of ([2]), 
not based on Theorem l4.4l (the proof of which is rather heavy)? Recall that C(s) is related to pattern avoiding 
permutations, since it counts r-avoiding permutations, for any pattern r of size 3. 

We conclude with a summary of the enumerative results obtained in [T] for R-equivalence classes and in 
the present paper for S-equivalence classes. 





All floorplans 


Guillotine floorplans 


R-equivalence 
classes 


Forbidden patterns: 

2-41-3, 3-14-2 


Forbidden patterns: 

2-4-1-3, 3-1-4-2 




• 


• 




• 


• 






• 

• 

• 

• 




• 

• 

• 

• 




• 


* 




• 


• 


Enumerating sequence: 

1,2,6, 22, 92, 422, 2074, 10754, . . . 
(Baxter numbers (30j A001181]) 
Growth rate: 8. 


Enumerating sequence: 

1,2,6,22,90,394,1806,8558,... 
(Schroder numbers [3(3 A006318]) 

Growth rate: 3 + 2\/2 * 5.8284. 


S-equivalence 
classes 


Forbidden patterns: 

2-14-3, 3-41-2 


Forbidden patterns: 

2-14-3, 3-41-2, 2-4-1-3, 3-1-4-2 






• 

• 




• 

• 










• 

• 




• 

• 




• • 
• • 
• • 
• • 


• 

• 








• 

• 


• 

• 








• 

• 


Enumerating sequence: 

1,2,6,22,88,374,1668,7744,... 
([301 A214358]) 

Growth rate: 4 + 2\/2 « 6.8284. 


Enumerating sequence: 

1,2,6,20,70,254,948,3618,... 
([301 A078482]) 

Growth rate: j2 * 4.5465. 

l-V8-s/2-ll 
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Appendix A. Proof of Proposition 1 3 . 1 2 L Paragraph D 



We prove that, given a permutation a, (any) floorplan P obtained by the construction described in Para- 
graphs A-B of the proof satisfies S(P) = a. 

In order to prove this claim, we will show that for all 1 < i < n, the segment Ki+i is the immediate successor 
of Ki in the \ order, and that K a -i^ i+ i^ is the immediate successor of K a -\^ in the / order. 

Let us first prove that the first statement implies the second. Let a' be obtained by applying a quarter-turn 
rotation p to a in counterclockwise direction. Let us denote by K[ the segment of P' = p(P) containing the 
point (£, cr'(i)). By Observation 13.131 the floorplan P' is associated with a' by our construction. That is, 
p(K a -i(i)) = K' n+1 _i. By assumption, K' n+1 _i = p{K a -i^)) follows K' n _i = p(K tT -i( i+1 )) for the \ order in 
P' . Applying the quarter turn clockwise rotation p~ x and the second remark following Proposition 12.12"! this 
means that K a -i^ + u follows K^-ir^s for the / order in P. 



Thus we only need to prove that Ki+i is the immediate successor of Ki in the \ order. By Observation ^. 10l 
the immediate successor of a horizontal (respectively, vertical) segment / in the \ order is R(/), LVB(I) or 
LHB(I) (respectively, B(7), UHR(I) or UVR(J))I 1C I depending on the existence of these segments and the 
type of joins between them. 

There are 8 cases to consider, depending on whether o~(i) < a(i + l) or a(i) > cr(z + 1) , and on the directions 
of Ki and Ki+i. 

Case 1: a{i) < o~(i + 1), Ki and Ki+i are vertical. 

Assume that Nj bounds Ki from above. Then, as shown in Paragraph C.l above, Kj is horizontal; 
furthermore, Ki and Kj have a T join at the point (i,cr(j)). In particular, the rightmost point of Kj has 
abscissa at least i + 1. 

If er(j) < a(i + 1), then Ni+i bounds Kj from the right. There is a n join of Kj and Ki+i at the point 
(i + l,a(j)) (Fig.EHl)). 

If > a(i + 1), then Nj bounds Ki+i from above and there is a T join of Kj and Ki+i (Fig. I25T 2)). 
If Nj does not exist and Ki reaches the upper side of the boundary, then no point can bound Ki+i from 
above, and, thus, K i+1 reaches the boundary as well (Fig. I25T 3)). 
In all these cases, it is readily seen that Ki+i is WK(Ki) . 

By Observation ^. 10[ UVR(i^i) is the successor of K i: unless XJHR(Ki) does not exist, B(A'i) := K p exists 
and its join with WK(Ki) is of type H (Fig. But this would mean that p < i, and the positions of N 

and N p would then contradict Observation ^. 141 



® 
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i i+1 
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/ i+l 
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Figure 25. The first case. 



Case 2: o~(i) < a(i + 1), Ki is vertical and Ki+i is horizontal. 



10 This notation is defined before Observation 12. 101 
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The point Ni bounds Ki+i from the left. Therefore, there is a I- join of Ki and Ki+i at the point (i, a(i + 1 ) ) , 
and K-i+i is a horizontal right-neighbor of Ki. Moreover, if Kk is another horizontal right-neighbor of Ki, then 
a(k) < a(i + 1): otherwise Ni cannot be a SW-neighbor of N k (Fig. 1251 left). Therefore, K i+1 = UHR(iQ). 

By Observation l2~T0l UHR(A^) is the successor oi K u unless UVR(A 4 ) := K p exists (Fig. [Ml right). If this 
were the case, K p and Ki+i would have a 1 join, and the position of Ni+i and N p would then be incompatible 
with Observation 13.141 



i i+l k 



i i+l p 



impossible 



Figure 26. The second case. 



Case 3: a(i) < a~(i + 1) , Ki is horizontal and Ki+i is vertical. 

We claim that this case follows from the previous one. Let a' be obtained by applying a half-turn rotation 
p to (the graph of) a. By Observation 13.131 the floorplan P' = p(P) is associated with a'. The points and 
segments p(Ni), p(N i+ i), p(Ki), p(K i+ i) in P' are in the configuration described by Case 2, with p(N i+ i) to 
the left of p(N). Consequently, p(N) is the successor of p(N + i) in the \ order in P'. By the first remark 
that follows Proposition [2T2l A^+i is the successor of Ni in the \ order in P. 

Case 4: a(i) < o~(i + 1) ; Ki and Ki + ± are horizontal. 

If this case, Ni bounds Ki+i from the left. Therefore, Ki must be vertical (see Paragraph C.l above). 
Hence, this case is impossible. 

Case 5: o~(i) > o~{i + 1), Ki and Ki+i are vertical. 

Since Ki+i is vertical, Ni+i has at most one NW-neighbor, which is then iVj. By Paragraph C.l above, Ki 
is then horizontal. Thus this case is impossible. 

Case 6: a(i) > o~(i + 1), Ki is vertical and -fQ+i is horizontal 

Since the segment if, is vertical, the point N has at most one SE-neighbor, which is then A^+i. Therefore, 
Ni+i bounds Ki from below, and there is a 1 join of Ki and Ki+i at the point (i,cr(i + 1)). In particular, 
K i+1 =B(Ki). 

By Observation 12.101 B(Ki) is the successor of Ki, unless UHR(A'j) := K k exists (Case (3.2) in Fig. HJ, 
or UllR(Ki) does not exist, but UVR(/\ i ) := K p does and forms with B(A^) a 1 join (Case (4.1) in Fig. H}. 
In the former case, K k reaches Ki and thus is bounded by Ni on the left, but then N and N + i are two 
SW-neighbors of Nk, and Kk cannot be horizontal. In the latter case, K p and -fQ+i would form a 1 join, and 
the positions of N p and Ni+i would contradict Observation 13.141 

Case 7: a(i) > a(i + 1), Ki is horizontal and Ki+i is vertical. 

This case follows from Case 6 by the symmetry argument already used in Case 3. 

Case 8: a(i) > a(i + 1), Ki and A';+i are horizontal. 

The point that bounds Ki from the right, if it exists, lies to the NE of Ni+\. Thus the abscissa of the 
rightmost point of Ki is greater than or equal to the abscissa of the rightmost point of K i+1 . 
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We will show that -Kj+i = LHB(ifi). Once this is proved, Observation 12.101 implies that LHB(A'j) is the 
successor of Ki, unless LVB(il'i) does not exist, but R(Ki) exists and forms with Ki+\ a 1 join (Case (4.2) 
in Fig. [5]). But this would mean that Ki+i ends further to the right than Ki, which we have just proved to 
be impossible. 

So let us prove that Ki+\ = LHB(iQ). We assume that Ki does not reach the left side of the boundary, 
and that -fQ+i does not reach the right side of the boundary (the other cases are proven similarly). Let Nk 
be the point that bounds Ki from the left, and let N m be the point that bounds iQ+i from the right. 

Consider A, the leftmost rectangle whose upper side is contained in Ki. The left side of A is clearly 
contained in Kf~. We claim that the lower side of A is contained in Ki+i, and that the right side of A is 
contained in K m . Note that this implies ifj+i = LHB(i^j). 

Let K p (respectively, K q ) be the segment that contains the lower (respectively, right) side of A. Clearly, 
q > k. If q < i, then K q is a vertical below-neighbor of Ki, and the positions of N q and TV, contradict 
Observation 13.141 Therefore, q > i + 1. 

Consider now the segment K p . Clearly, a(p) > a(i + 1). One cannot have p > i + 1: otherwise TVj+i (or a 
point located to the right of Ni+i) would bound K p from the left, and K p would not reach K^. One cannot 
have either p < i: otherwise TV, (or a point located to the left of Ni) would bound K p from the right, and K p 
would not reach K q . Since p ^ i, we have proved that p = i + 1, and K^ is bounded by Ni + i from below. 

Finally, K q coincides with K m : otherwise, q <m, and K q is a vertical above- neighbor of Ki+i; however, in 
this case N q would bound Ki + \ from the right, and Ki+i would not reach K m . 

We have thus proved that Ki+i = LHB(Ki), and this concludes the study of this final case, and the proof 
of Proposition 13. 121 

Appendix B. Proof of Lemma 14.51 

Let Ni = (i,j) be inside a maximal F-block of R(P). Assume for the sake of contradiction that TV; is 
strong, and for instance, has several NE- neighbors. Let Nk be the leftmost NE- neighbor of Ni, and let Ng be 
the lowest NE- neighbor of TVj. If k > i + 1, then a(k - 1) < cr(i), and, therefore, i,k - l,k,£ form a forbidden 
pattern 2-14-3. Thus k = i + 1. Symmetrically, a(l) = j + 1 (Fig. [27]). Note also that a(i + 1) > j + 1 and 
cr _1 (j + 1) > i+ 1. Since the points of S(P) inside an F-block are either on the diagonal or the anti-diagonal 
of this block, TVj is the highest (and rightmost) point of S(P) inside the maximal F-block that contains it, 
and this F-block is of ascending type. In particular, either p(i + 1) = j + 1, or p(i) = j + 1 and p(i + 1) = j. 

Since p(i + l) <j + l and a(i + l) > j + 2, then p{i + 2) > j + 3 (Theorem l4.2[) . Symmetrically, p~ 1 (j + 2) > i + 3. 
But then the position of the point (p _1 (j+2), j+2) is not compatible with the position of TVj + i: by Theorcm l4.2l 
there cannot be a point of p located to the right of p(i + 2) and in the rows between those of p(i + 1) and 
TVj+i. Hence TVj cannot have several NE-ncighbors. Symmetric statements hold for the other directions, and 
TVj is a weak point. 

Now let TVj = (i,j) be a point of the graph of a, not inside a maximal F-block. Assume without loss of 
generality that p has an ascent at i: p(i) <j< p(i + 1) and (by Theorem 14. 2p p~ x (j) < i < p~ l (j + 1). We shall 
show that TVj has several SW-neighbors or several NE-neighbors. We denote Mj = (i,p(i)). 

First, if p(i) = j and p(i + 1) = j + 1, then Mj and Mj+i form an F-block, and TVj is inside this block. 
Therefore, cither p(i) ^ j or p(i + 1) j + 1, and we may assume without loss of generality that p(i) + j; hence 
p(i) < j and p _1 (j) < i- Then it follows from the definition of Baxter permutations that p(i-l) < j (otherwise, 
there is an occurrence of 2-41-3 at positions p~ l {j),i - p (j + 1)). Consequently, we have a(i-l) < j -1. 
Symmetrically, /0 _1 (j - 1) < i and <t _1 (j - 1) < i - 1. There are two possibilities: either a(i - 1) < j - 1 and 
<T _1 (j - 1) < i- 1, or a(i - 1) = j - 1. In the former case, TVj_i and N^-irj-u are two SW-neighbors of TVj, and 
we have proved that TVj is strong. Let us go on with the latter case, where p(i - 1) = j and p(i) = j - 1. 

If p(i + 1) = j + 1, then Mj_i, Mj and Mj + i form an F-block, and TVj is inside this block, which contradicts 
our initial assumption. Otherwise, p(i + 1) ^ j + 1, and an argument similar to the one developed just above 
shows that either TVj+i and TV CT -i(j + i) are two NW-neighbors of TVj, or p(i + 1) = j + 2 and p(i + 2) = j + 1. 
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Figure 27. Some points of the combined diagram of p and a. The grey points represent 
the two possibilities p(i + 1) = j + 1, or p(i) = j + 1 and p(i + 1) = j. 



In the latter case, Mj_i, Mj, Mj+i and Mi+2 form an F-block containing iV», which contradicts our initial 
assumption. 



Let a be a (2-14-3, 3-41-2)-avoiding permutation of size n. Let B be the set of Baxter permutations 
whose S-permutation (described by Theorem 14. 2 1) is a. Lemma T4.5I determines which points of the graph of 
a are inside an F-block. These points are organized along the diagonal or anti-diagonal of their blocks. It 
follows that the location of all non-trivial F-blocks in the graph of p, for p e B, and their type (ascending or 
descending, for blocks of size at least 3), are also determined uniquely. It remains to show that the location 
of the trivial F-blocks (that is, F-blocks of size 1) is also determined by a. 

Assume for the sake of contradiction that B contains two distinct permutations p\ and pi. Let i be the 
abscissa of the leftmost trivial F-block that is not at the same ordinate in the graphs of pi and p2 . Denote 
j = <r(i). By symmetry, we only have to consider two cases: (1) pi(i) < P2{i) ^ j] (2) Pi{i) < j < P2{i)- 

In the first case (Fig. [28)) . denote k = P2(i)- Consider pi 1 (k). By assumption, p\ l {k) + i. Since pi(i) < k 
and a(i) = j > k, we have Pi 1 (fc) < i by Thcorcm l4.2l However, this is impossible since the F-structures of p\ 
and p2 coincide to the left of the ith column. 

Consider the second case, pi{i) < j < pi(i) (Fig. [29]). Since pi(i) < j and a(i) = j, the areas [1, i] x {j + 1} 
and [i + l,n] x {j} are empty in the graph of p\. Similarly, since P2(i) > j + 1, the areas [1, z] x {j} and 
[i + l,n] x {j + 1} are empty in the graph of p2- Since the F-structures of p\ and p2 coincide in 1] x [l,n] 

the areas [1, i- 1] x {j, j + 1} are empty in the graphs of both permutations. Given that rows cannot be empty, 
this forces pi(i) = j and p%(i) = j + 1 (Fig. l30|) . 

Assume without loss of generality that a(i + 1) < j. Since pi(i) = j and <r(i) = j, we have, by Theorem l4.2l 
Pi(i + 1) > j + 1. In fact pi(i + 1) > j + 1 since otherwise the point (i,pi(i)) would not form a trivial F-block. 
Now, since <j{i + 1) < j, the area [i + 2, n] x {j + 1} is empty in the graph of p\. This area is also empty in 
the graph of p2, since P2{i) =3 + 1- Since the F-structures of p\ and p 2 coincide in [l,i - 1] x [l,n], the area 
[1, i - 1] x {j + 1} is also empty in the graph of p±. Since pi(i) = j and p\(i + 1) > j + 1, we have a contradiction: 
the whole row j + 1 is empty in the graph of p\ . 

Thus, we have proved that all p e B have the same F-structure. 



Appendix C. Proof of Theorem 14.41 the "only if" direction 



ORDERS INDUCED BY SEGMENTS IN FLOORPLANS 
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Figure 28. Proof of Theorem 14.41 the case pi(i) < P2(i) ^ j- The shaded areas contain no point. 
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FIGURE 29. Proof of Theorem 14.41 the case < j < p%(i)- 
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FIGURE 30. Proof of Theorem 14.41 the case p\{i) < j < P2(i), continued. 
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